So the hybrid cloud rabbit hole has no end, but luckily it is full of surprises and wonders.

Comms

Applications need to communicate with one another in just about any non-monolithic business environment, and horizontal scalability requires the opposite of monolithic design. Luckily there are lot’s of mature tools to help tackle the problem of gluing back together a monolith once it’s been decomposed into micro-services.

Open Source() NATS is a secure by default, massively horizontally scalable, topology agnostic, messaging service system similar to an enterprise message bus. It can handle streams and queues, pub/sub relationships, supports playback and arbitrary retrieval. Most importantly it does this securely by default. As such this is a fundamental connectivity building block for decomposed applications as well as an empowering moderator for cross communication of monolithic applications. With growing support for Kafka streams integration this is a system going in the right direction. This is a mature and very capable system for communicating inside a zone of control, event if that zone of control extends over an arbitrary number of underlying physical systems and places.

Mixed Source() Kong(Konnect) on the other hand is a communication gateway and service mesh manager. The API capabilities of Kong are impressive, from service catalogues and developer portals to built in analytics and TDD support. The API gateway serves as a system for exposing systems to external interaction from the zone of control. As an externally facing gateway this is a critical security enforcement point.

Storage

Another key bottleneck to scalability is the move from vertically scaled storage solutions to horizontally scaled solutions. Distributed systems are generally better but potentially more complex so it’s a great thing when the community has already solved the complexity problems for you.

Minio is an object storage application that is compatible with the AWS S3 API.

OpenIO

Ceph

Task Queues

Sometimes you need either simple queue services or dedicated services to tie things together. There are numerous options here, this list that is weighted towards relative ease in deployment across clouds.

Kafka is one of the most advanced open source queue systems, so much so that it calls itself a stream system. Streams in this sense are just queues that never stop. It has a full feature set, horizontal scalability, multi-region redundancy, sticky message partitions, advanced pub/sub, storage and processing.

NSQ

RabbitMQ

ZeroMQ

Celery For a simple queue service that you can run across multiple clouds Celery is one of the easiest to use and configure.